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FOREWORD 



This technical report su mm arizes the progress and preliminary analyses to date for McREL’s study of 
academic success in high-needs elementary schools. Included are descriptions of sampling, participant 
recruitment, and data collection procedures; summaries of measurement viability findings; and a 
discussion of data analyses plans. The report concludes with an outline of next steps to project completion 
planned for 2005. 

Last year at this time, McREL submitted the design for this phase of the work which extends a line of 
research begun as pilot studies in 2001. Additionally, it included the review of four sets of research 
literature, each related to a different component of school effectiveness. Integrating the four sets of 
literature, the team developed a comprehensive model of academic success in high-needs elementary 
schools. Reviewers of the proposal affirmed the value of the research and the soundness of the research 
design and made useful suggestions for several of the model subcomponents and sampling procedures. 

In 2004, the reviewer comments were incorporated into the model, the data collection, and sampling plan. 
Pools of schools from which to recruit study participants were identified, participants recruited, data 
collected, and preliminary measurement viability analyses were conducted. Since data collection is still in 
progress, there are no findings to report yet. Thus, this report serves as an internal document reporting on 
progress in 2004. As such, the intended audience for this report is primarily the research team, providing 
an opportunity to take stock of the work in progress, and IES staff. 
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INTRODUCTION 



The goal of the No Child Left Behind Act parallels what educators have long set their 
sights on: to equip every child with the knowledge and skills necessary for success in 
future schooling and in life (Cicchinelli, Gaddy, Lefkowits, & Miller, 2003, p. 7). 

The goal of the No Child Left Behind (NCLB) Act of 2001 is for all students to demonstrate proficiency 
in reading and mathematics by 2014. The goal is to be met incrementally by students achieving a 
prescribed level of progress each year. In 2002-2003, over 25,000 schools did not reach their target for 
annual yearly progress (AYP) (Quality Counts, 2004). To expand the research base available to educators 
to guide their efforts to improve student achievement in high-poverty schools, Mid-continent Research for 
Education and Learning (McREL) is conducting a multistate study of academically successful high-needs 
schools. 

This report provides a summary of progress to date on the study of academic success in high-needs 
schools and is intended for use by the research team and IES staff. The report begins with a history of the 
project’s origins and a brief review of pilot study findings. The Introduction concludes with an overview 
of the conceptual framework and the research design. The following section summarizes project 
accomplishments during 2004 in three areas: sampling, data collection, and measurement viability. In the 
third and final section of the report, a brief status update and steps to project completion are presented. 

Project Origins and Pilot Study Findings 

This project was launched in response to the needs of educators for research-based knowledge about 
academic success in high-poverty schools. Pilot studies were conducted in 2000 and 2001 in 18 
elementary schools from two Central Region school districts. All schools in the study were identified as 
“high needs” based on the criteria that 50% or more of students enrolled were eligible for free or reduced- 
price lunch. The sample was divided into two groups — one is high performing (with a percentage of 
students demonstrating proficiency in reading and mathematics at or above state average) and the other 
low performing (with a percentage of students demonstrating proficiency in reading and mathematics 
below state average). Several features distinguished the high-performing (HP) and low-performing (LP) 
high-needs schools. 

• Teachers in HP schools demonstrated a stronger sense of responsibility for student 
learning than teachers in low-performing (LP) schools (Lauer, 2001). 

• High-performing (HP) schools provided teachers with Significantly more 
professional development focused on content standards and diverse learners 
compared to LP schools (Lauer, 2001). 

• Teachers in HP schools compared with LP schools more frequently reported using 
adaptive instructional practices, including leveled books in reading, as well as 
tutoring to individualize instruction in both mathematics and reading (Akiba & 

Apthorp, 2003; Apthoip, 2002). 

The distinguishing features of high-performing, high-needs (HPHN) schools in this study — a strong 
sense of responsibility for student learning, attention to diverse and individual student learning profiles, 
and an emphasis on content-focused professional development— are consistent with school-level 
correlates of academic achievement found in other research (Garet, Birman, Porter, Desimone & Herman, 
1999; Goddard, 2001; Hill & Rowe, 1998). 
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Conceptual Framework and Research Design 



Subsequent to the pilot studies in 2002-2003, a more systemic view of school success was adopted and 
four sets of research literature were reviewed and integrated to propose a comprehensive model of how 
schools are organized for academic success. The model included four key components and posited 
relationships between the components and a school’s academic performance. The model’s 
conceptualization and research base were presented in this study’s design proposal prepared last year 
(Apthorp, 2003). Figure 1 shows the second iteration of the model as revised in response to reviewer 
comments. 
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Figure 1. Conceptual model of academic success in high-needs elementary schools. 

As shown in Figure 1, the four key components of the model are School Environment, Leadership, 
Professional Community, and Instruction. 1 In 2003, a nonexperimental quantitative comparative study 
design was proposed to examine the adequacy of the model (Apthorp, 2003). This design is appropriate 
for studying relationships between a presumed cause and effect when both are identified and measured 
but in which other structural features of experiments are missing, such as random assignment (Shadish, 
Cook, & Campbell, 2002). The design for this project addresses two main research questions which are: 



1 Changes from the original model are in how we define Instruction. Whereas in the original version (see Apthorp, 2003), 
Structure and Individualizing were not differentiated, in the current model, each are considered separate subcomponents. 
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1. How do each of the four key components of Leadership, Professional Community, School 
Environment, and Instruction contribute to school-wide performance in high-needs schools? 

2. What is the configuration of relationships among the key components that differentiates high- 
performing, high-needs schools from low-performing, high-needs schools? 

External reviewers provided feedback on the 2003 research proposal, affirming the general soundness of 
the design, identifying particular aspects of model components that needed clarification, and reiterating 
the importance of verifying assumptions about the two-group sample proposed for the comparative 
design. In 2004, we incorporated reviewer suggestions into our sampling procedures and model and 
instrument development. We established a two-group sample, developed and refined a Teacher and 
Principal Survey, and collected data. The procedures and results of each of these tasks arc described in the 
following sections of this report. 

PROJECT ACCOMPLISHMENTS 2004 

In 2004, sampling procedures were implemented and survey instruments were developed. Pools of HP 
and LP high-needs schools were identified, study participants recruited, teacher and principal data were 
collected, and measurement viability examined. In this section, challenges and accomplishments with 
regard to implementation of each of these aspects of the project design are reported. 

Sampling 

Having selected a nonexperimental quantitative comparative study design, a multistage sampling process 
was proposed (Apthoip, 2003). This process was used to (a) identify high-performing (HP) and low- 
performing (LP) high-need schools, and (b) create two demographically comparable groups of schools 
(HP and LP). 

High-need schools were defined as high poverty, where high poverty was operationalized as 50% 2 or 
more of students were eligible for free or reduced-price lunch (FRL). To identify a pool of high-need 
elementary schools, states for which fourth-grade assessment data in both reading and mathematics were 
available across three consecutive years were examined first (2000, 2001, and 2002). Eleven states met 
this criterion (Colorado, Ohio, Georgia, Kansas, Michigan, Minnesota, Missouri, New Jersey, Oregon, 
Virginia, and Texas). Using the high-needs elementary school populations within each state, a multiple 
regression analysis was conducted to identify pools of HPHN and LPHN schools. Finally, individual 
schools were recruited to participate from each state pool of HPHN and LPHN schools. 

Identification of HPHN and LPHN Schools 

To prepare data for identifying HP and LP high-needs schools, several steps were used. School 
performance status was calculated (HP or LP). That is, the average percentage of students scoring 
proficient and above in reading and mathematics in each school identified as high-needs for each of three 
years (2000, 2001, and 2002) was determined. Correlation coefficients between percentage of students 
proficient in reading and mathematics were computed to assure that collapsing across reading and 
mathematics did not mask uneven levels of performance in a given year. In each case, these correlation 
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Other researchers define high poverty as 75% or more FRL students (e.g.. Puma et al., 1997); however. Puma's more stringent 
definition unduly constrains the sample. A criterion of 50% or more FRL students, nationwide, encompasses 35% of all 
elementary schools (U.S. Department of Education, National Center for Education Statistics, 2003). A criterion of 75 percent or 
more FRL students encompasses only 18% of all elementary schools. 
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coefficients were significant ranging from a low of .55 to a high of .93 (see Table A-l in Appendix). This 
suggests that by collapsing performance across two core subject areas, the criterion variable (i.e., school 
performance) reflects a uniformly high or low level of performance, not that of reading or mathematics 
alone. 

To identify HPHN and LPHN schools, two school composition valuables and one community valuable 
were used as predictors of performance status in a multiple regression. Schools whose performance was 
above (HP) or below (LP) the level predicted were labeled as high- and low-performing, respectively. The 
predictor variables were selected based on the findings of prior research that identified them as strong 
predictors of achievement (Heck, 2000), including socioeconomic status (the percentage of students 
eligible for free or reduced-price lunch (FRL), percentage of minority students, and locale. 3 School data 
on FRL (2000, 2001, 2002), percentage of minority students (2000, 2001, 2002), and locale were 
collected from state Departments of Education or in some cases from Common Core data. The average 
percentage of FRL and the average percentage of minority students were calculated across these three 
years. 

The regression formula was applied to the high-needs schools in each of the 1 1 states. To identify schools 
whose performance was above or below the level predicted by the regression, a +.75 residual was used to 
define HPHN schools and a -.75 residual to define LPHN schools. Use of + .75 residuals as the cutoff 
points has been recommended for creating consistent school effectiveness indices (Crone & Teddlie, 
1995). The last step was to eliminate the bottom one third of schools from the HPHN group and the top 
one third of schools from the LPHN schools based on achievement levels in order to minimize potential 
overlap. This process yielded a pool of 739 HP and 738 LP schools across the 1 1 states (for number of 
schools in each state pool, see Table A-2 in Appendix). 

Participant Recruitment 

In each school, all teachers and the principals were invited to participate in either the teacher or principal 
survey. Lor the puiposes of this study, teachers were defined as professionals who provide student 
instruction as their primary responsibility in either classroom or small group pull-out sessions. This 
includes special teachers (e.g., music, PE, computer lab, etc.), English as a Second Language (ESL) 
teachers, Title I teachers, reading teachers, reading or mathematics coaches, and long-term substitute 
teachers. This definition does not include professionals who provide related and/or health services, such 
as school psychologists, nurses, guidance counselors, speech and language pathologists, or parent 
liaisons/coordinators. 

Participant recruitment began in December 2003 in five of the 1 1 states and proceeded in three stages. 
Lirst, district superintendents were contacted through a mailing which was then followed up by phone 
calls to inform them of the purpose and nature of the study and to ask their approval to contact principals 
directly. Our correspondence included a list of the possible high-needs elementary schools in the 
superintendent’s district, but did not identify performance categories (HP or LP). Second, and with 
approval from the district superintendent, 4 principals were invited to participate. Principals either agreed 
or declined the offer. Participating principals were asked to nominate a site liaison who assisted with data 
collection. Third, individual teachers were asked for their informed consent to participate. 
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Other variables which describe the composition of a student population and predict achievement include prior achievement, 
gender, language background, and proportion of special education students (Heck, 2000). The possibility that these variables 
serve as alternative explanations of observed performance levels will be considered during the data analysis process. 

4 When required, approval to conduct the study also was ought from district research review committees. 
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Study participation was voluntary, although follow-up opportunities were offered and incentives provided 
to encourage participation, including: 



• an opportunity to contribute evidence to research-based knowledge; 

• a minimal data collection burden (no survey distribution; no classroom observations 
or administration of student assessments; a one-time survey to be completed online at 
the respondent’s convenience); 

• a summary report of the study’s findings; 

• a $30 gift certificate from a national book store for site liaisons; 

• a chance for teachers to win a gift certificate from a national book store; 

• a $1 gift certificate per teacher to local fast-food restaurant; and 

• copies of selected McREL professional publications. 

To encourage high-response rates per school, graduated incentives were provided. At minimum, the site 
liaison’s $30 gift certificate was made contingent on a 75% response rate. Response rate was defined to 
include teachers who declined to complete the survey as well as those who agreed to complete the survey. 
The second-tier incentive was contingent on an 80% survey completion rate (not counting recorded 
decisions not to participate ). To schools reaching the 80% survey completion rate, two professional 
products valued at $70 each were provided: Teaching Reading in Social Studies by Doty, Cameron and 
Barton (2003) and Teaching Reading in Mathematics by Bart cm and Heidema (2002). The goal for within 
school response rate was 70%. 

At each potential site location, principal recruitment began as soon as approval from the district was 
obtained. As soon as approval from the principal was obtained, teacher recruitment began. By April 2004, 
not enough superintendents and principals had agreed to participate as planned. 5 At least 30 to 50 
between-level units (schools) are recommended for multiple group, multilevel modeling (Muthen & 
Muthen, 2003). Therefore, a second recruitment effort was initiated during the summer of 2004 in the 
remaining six states (see Table A-2 in Appendix for a list of these six states). Some districts contacted in 
the spring required more time to respond and their schools became paid of the second set of schools. 

Thus, data were collected in two waves with two corresponding windows open for access to the online 
surveys. Initial recruitment efforts resulted in Wave I data being collected from March 1 to June 30, 2004. 
The second recruitment efforts will result in Wave II data collected from August 23 to October 31, 2004. 

Data Collection 

In both Wave I and II of data collection, the site liaisons selected by principals to assist with data 
collection supplied McREL researchers with a list of teacher and principal e-mail addresses. E-mail 
invitations and informed consent letters, with the Teacher Survey or Principal Survey, were sent 
electronically to each teacher and principal respectively. The content of the surveys used in both waves of 



5 The most common reason for declining to participate was a prior commitment to another data collection effort, usually part of 
ongoing school improvement initiatives (such as Reading First or Comprehensive School Reform) or for accountability or 
accreditation purposes at the state and/or national levels. 
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data collection was identical. All respondents were asked to answer questions about activities and 
practices at their school during the 2003-2004 year. The collection and analysis of Wave II survey data 
from teachers and principals is now nearing completion. 

In Wave I (March 1 to June 30, 2004), data were collected from 38 schools in five states (Michigan, 
Minnesota, Ohio, Oregon, and Texas). These data were supplied by 25 HP and 13 LP schools. The 
median number of total teachers per HP school was 21, ranging from 5 to 59. The response rate for 
teachers in the HP schools ranged from 6% to 100%, with a median of 57%. The median number of 
teachers per LP school was 22, ranging from 16 to 28. The response rate for teachers in LP schools 
ranged from 19% to 83%, with a median of 50%. 

In all, 449 teachers completed the Teacher Survey and 19 principals completed the Principal Survey. The 
majority of Wave I teachers (64%) are regular classroom teachers either teaching in the primary, 
intermediate, or all elementary grades (43%, 39%, and 18%, respectively). The majority (68%) holds a 
professional certification to teach and all of the respondents, except for one, hold a bachelor’s degree. In 
addition, 44% hold a master’s degree and 21% have an education specialist degree. The average number 
of years teaching is 14 years, with an average of eight years teaching in their current school. Of the 19 
Wave I principals who completed the Principal Survey, all hold a principal’s certification from their state 
and a master’s degree. None have doctoral degrees. On average, these principals had worked as principals 
for 12 years, and in their current schools as the principal, on average, for approximately seven years. 

Measurement Viability 

Because Wave II data collection and analysis are still in progress, Wave I data were used to examine 
measurement viability for representing model components and subcomponents. Measurement viability 
will be reexamined as Wave II data become available. Additionally, as Wave II data become available, 
assumptions about sample size and characteristics critical to the conduct and interpretation of analyses 
planned for examining the posited model will be reexamined. 

Constructs 

Teacher and principal surveys were developed with sections corresponding to each of the key components 
and related subcomponents identified in the model (see Figure 1). An extensive literature review informed 
how we crafted the scope and substance of item clusters in each section. A summary of the research 
literature, originally reported in the study design proposal (Apthorp, 2003), is presented below. In the 
summary, we define each of the key components, School Environment, Leadership, Professional 
Community and Instruction, and explain each one’s related subcomponents. 

School environment. School environment refers to those school-level variables that relate directly to the 
school environment and cannot be ascribed to a particular position (i.e., teachers, curriculum coordinators, 
or principals). Rather, these variables reflect policies created at the school, district, or community level 
that impact the entire school faculty, parents, and students. Four subcomponents define school context. 

Assessment and monitoring is defined as an evaluation policy and monitoring system used at all levels 
and includes testing, identifying learning problems, and providing remediation (Teddlie & Reynolds, 
2000; Creemers, 1994). This assessment and monitoring system helps teachers focus on important core 
goals, monitor progress, and provide remedial assistance (McCollum, 1995; Mortimore, Sammons, Stoll, 
Lewis, & Ecob, 1989; Taylor, Pearson, Clark, & Walpole, 2000). 

Academic press for achievemen t is defined as a school- wide orientation towards high expectations for all 
students. Academic press for achievement is evident in clear academic goals, appropriate use of 
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homework, and the use of records to monitor student progress toward mastering basic skills (Creemers, 
1994; Marzano, 2000). As an indication of the importance of academic press for achievement, teachers in 
effective schools spend substantially more classroom time on academic content than teachers in 
ineffective schools who instead spend more time managing nonacademic activities, such as preparing for 
recess and lunch and fundraising (Teddlie, Kirby, & Stringfield, 1989). 

Safe and orderly climate is one of the most important variables in helping low-achieving students succeed 
in school (Borman & Rachuba, 2001). When a school reports fewer incidences of disciplinary problems, 
there is a decrease in achievement gaps between white and minority students (Raudenbush & Bryk, 

1989). A school with an orderly climate has policies in place that clearly articulate rules and codes of 
behavior along with associated rewards and punishments. Faculty and staff practice “thoughtful 
prevention” of disruptions and are consistent in the way they enforce rules and punishments (McCollum, 
1995). This does not mean that the school has a strictly negative or severe environment but, rather, that 
positive and open interactions between staff and students are encouraged (Rutter, Maughan, Mortimer, & 
Ouston, 1979; Creemers, 1994; Flallinger & Fleck, 1996; Fleck, 2000; Marzano, 2000). 

Productive parental involvement refers to positive and productive relationships between the school’s staff 
and students’ parents. This is indicative of effective schools (Teddlie & Reynolds, 2000). Parents are 
involved in the school and their voice is present in the school culture and operating principles. In order to 
accomplish effective parental involvement, there must be good written exchange between schools and 
parents, a parent involvement policy, parent training, and easy access to administrators and teachers with 
an informal open-door policy (Mapp & Flenderson, 2002; Mortimore et al., 1989). 

Leadership. As portrayed in recently completed reviews, leadership has three critical effectiveness 
elements: setting directions, developing people, and redesigning the organization (Leithwood, Louis, 
Anderson & Wahlstrom, 2004; Marzano, Waters & McNulty, in press). Each of these elements is 
included in the model’s three subcomponents of Leadership as defined and explained below. 

Framing and communicating a common mission deals with setting direction. Leadership clearly defines 
goals that determine the areas in which school staff expend their resources. The focus is on linking beliefs 
and actions in the school, for example, academic expectations, opportunity to learn (OTL), and time for 
learning. The principal’s values and beliefs are known to teachers and are aligned with the mission and 
goals. The mission and goals are a prominent paid of the day-to-day operation of the school. School goals 
consistently show up as a significant factor contributing to principals’ impact on student achievement 
(Flallinger & Fleck, 1996). 

Manages instruction refers to developing and allocating the human and material resources necessary for 
student learning through governance, school climate and instructional organization (Fleck, Larson & 
Marcoulides, 1990). It includes monitoring and supporting the day-to-day work of teachers in classrooms 
and ensuring that curriculum, assessment, and instruction are aligned. The principal must be visible and 
supportive in classrooms, maintain information on student performance that can translate into school and 
classroom practices, and make available and effective use of instructional support personnel (Levine & 
Lezotte, 1990). 

Redesigning the organization is a function of leadership that is necessary in a high-needs school in order 
to “beat the odds.” This involves modifying school policies, structures, and culture to enable change. The 
principal must draw on a wide array of knowledge, skills, and tools to accomplish change: incentives and 
disincentives to make changes aligned with high performance, knowledge and ability to communicate the 
research basis for changes, willingness to take risks and support teachers in trying new things, and the 
ability to generate quick wins and sustain the long march to improvement. Levine and Lezotte (1990) also 
suggest that redesigning the organization involves vigorous selection and replacement of teachers, 
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“maverick orientation,” and buffering (i.e., stepping into the middle to release tension, high expenditure 
of time and energy for school improvement actions, and acquisition of resources). 

Professional community. Professional community is defined by shared norms and values, collective 
focus on student learning, collaboration, deprivatization of practice, and reflective dialogue (Louis, 

Marks, & Kruse, 1996; Newmann & Wehlage, 1995; Newmann, King, & Secada, 1996). "These elements 
do not constitute a hierarchy, and school-wide professional community demands at least a minimal level 
of each of the elements." (Louis, Marks, & Kruse, 1996, p.760). From this and other research on 
professional community, professional community is defined by three subcomponents: professional 
development, collaboration and deprivatization, and support for teacher influence. 

Professional development within a community of learners as an important aspect of professional 
community (Smylie, Allensworth, Greenberg, Harris, & Luppescu, 2001). Research supports the 
importance of two quality features: collective sustained participation (Garet, Birman, Porter, Desimone & 
Flerman, 1999; Newmann & Wehlage, 1995) and reflective dialogue. Reflective dialogue allows teachers 
to think about, analyze, and share knowledge related to instruction, curriculum, and student learning, 
which leads to a deeper understanding of teaching (Louis et al., 1996; Secada & Adajian, 1997; Bryk, 
Camburn, & Louis, 1999). 

Collaboration among teachers fosters the sharing of work and expertise, as well as a sense of affiliation 
and support (Louis et al., 1996; Secada & Adajian, 1997; Newmann & Wehlage, 1995). Work culture is 
more collaborative in high-performing compared to low-performing schools (Bruner & Greenlee, 2000). 
Research also indicates that teachers in professional communities are encouraged to work together in one 
another’s classrooms ( deprivatization of practice) by interacting with one another, observing, mentoring, 
providing feedback, and sharing expertise (Louis et al., 1996; Bryk et al., 1999). 

Support for teacher influence refers to teacher empowerment. Principals and administrators share 
leadership with staff and create shared ownership with teachers around norms, values, mission, and 
expectations (Fiord, 1997; Louis et al., 1996). In order to accomplish this, democratic school structures 
are put in place and teachers have freedom and influence to respond to issues and offer input (Newmann 
& Wehlage, 1995, as cited in Louis et al., 1996). A climate of respect from the community, colleagues, 
and leaders for teachers’ input regarding the learning environment supports and sustains this element of 
professional community (Louis et al., 1996; Bryk et al., 1999). 

Instruction. In a standards-based system, instruction is defined by three subcomponents: structure, 
individualizing, and challenging opportunities to learn. Structured teaching clarifies for students what 
they should know and be able to do and whether or not they are progressing toward these goals. 
Individualizing instruction allows every student to begin where they arc and learn what they need to 
become proficient. Challenging opportunities to learn reinforce high expectations and prepare students for 
unanticipated circumstances in later schooling and in life. 

Individualizing instruction involves making a concerted and caring effort to assist rather than dismiss 
struggling learners (Weber, 1971). Individualizing instruction appropriately is closely related to the 
concept of effective learning time. When time spent on learning is equal to time needed to learn, time is 
used effectively and achievement growth is likely. As shown in Flill and Rowe’s (1998) research, 
“students make the greatest progress when instruction is pitched at the right level for each individual” (p. 
327). Syntheses of research on effective schools confirm that individualizing instruction is a significant 
correlate of academic achievement (Schreerens & Bosker, 1997; Waxman, Wang, Anderson & Walberg, 
1985). 




Structure in teaching involves communicating clear learning goals, directing student attention to the 
goals, modeling, scaffolding and cuing student approximations, encouraging students’ active engagement, 
detecting and correcting errors or misconceptions, and appropriately reinforcing students to confirm that 
what they have learned is important. Research across core subjects in elementary schools, in general and 
in schools serving low-income students in particular, has consistently shown that a structured approach to 
teaching is associated with greater achievement (Baker, Gersten & Lee, 2002; Good & Brophy, 1986; 
Fraser, Walberg, Welch & Flattie, 1987; Fleistad, 1997; National Reading Panel, 2000; Walberg, 1990). 

Challenging opportunities to learn refers to students’ frequent engagement in higher-order thinking and 
problem solving activities. Researchers Teddlie and Reynolds (2000) found that the ability to instill in 
students a belief that they could learn was critical to the success of low-SES, effective schools. Balancing 
attention to mastery of basic skills with attention to development of advanced or higher-order thinking is 
important (D’Agostino & Fliestand, 1995; Knapp & Associates, 1995; Lauer, Palmer, Van Buhler & 

Fries, 2002; Puma, Karweit, Price, Ricciuti, Thompson, & Vaden-Kiernan, 1997; Wenglinsky, 2003). 

Clusters of survey items were crafted from the literatures summarized above to reflect the subcomponents 
of each key component. The School Environment and Leadership components are addressed in both the 
Teacher and Principal survey. The Professional Community and Instruction components are addressed 
only in the Teacher Survey. Item format for the model component sections is a Likert-type scale with 
responses representing “to what extent” or “agree/disagree” continuums. Survey directions ask 
respondents to answer the questions in terms of activities and practices at their school during the past 
school year (2003-2004). 

An additional section in each survey asks about the respondent’s background, including education level, 
areas and levels of certification, and years working in their respective position and school. In addition, the 
Principal Survey includes a section that asks about school funding sources and school programs, in 
particular, the use of school-wide reform strategies, preschool, tutoring, reading and math 
curricula/programs, and professional development. Item format for the background and school program 
sections require yes/no responses, “check all that apply” or fill in the blank. 

In December 2003, field tests of each survey instrument were conducted to determine length of time to 
complete and evaluate and improve clarity of item wording. In January 2004, 60 teachers in four high- 
needs elementary schools participated in a pilot-test of the Teacher Survey. Reliability analyses on each 
subcomponent set of items showed adequate internal consistency (i.e., coefficient alphas were .70 or 
greater) except for one subcomponent. Item revisions were made to improve the one subcomponent set of 
items prior to collecting Wave I data. 

Procedures and Results 

For the Wave I data from the teacher and principal survey, classical item analyses were conducted using 
LERTAP (Nelson, 2000) for the total set of items and for each intended subset of items. At the item level, 
results were examined to assess the distribution of responses across the response scale. To address the 
possibility of differential effects of a systematic positive response across items, responses were 
transformed to nine category z-scores for each item based on the mean and standard deviation. This 
transformation allowed interpretation along a common scale and ensured that each item reflected the 
equal weighting in the aggregation for each subscale as intended, based on the current research in this 
area. Additionally, this helped to ensure the intended placement of the subscales within composite scales. 
The appropriate subscale scores were then summed to create each of the composite scale scores. 

Appropriate use and interpretation of the results of any measurement depend on the extent to which the 
resulting data reflect the intended construct rather than extraneous characteristics or random variability. 
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For the intended use of survey results in this study, a primary concern is the appropriate 
operationalization of the underlying constructs. The intended scope and content of the sets of survey 
items, the clustering of items into subscales, and the weighting of the subscales for the composites were 
derived from extensive review of the research literature as summarized above. Based on this information, 
items were equally weighted within subscales and subscales were summed to yield an unweighted 
composite. 

Generally, reliability is deemed a prerequisite condition for the relevance of a measurement; thus, 
measures exhibiting low reliability limit the relevance, usefulness, and validity of the results. Because the 
magnitude of the differences, if any, is unknown for the populations represented by this study, 
measurement precision and reliability were intended to be maximized in order to be sensitive to small 
variations in the constructs to be measured. While certain indices of reliability include estimates of 
variation over time/occasions as well as across items. Coefficient Alpha is often used for surveys such as 
those employed in this study. This index theoretically ranges from 0.00 to 1.00 and enables calculation of 
the standard error of measurement (s.e.rn.), which provides an estimate of the magnitude of the “grain of 
salt” to use when interpreting composite scale and subscale scores. 

Teacher survey reliability and validity. Information about the reliability of the teacher survey results for 
the 449 responding teachers is provided in Table 1. As shown in Table 1, all four composite scales 
evidenced high levels of reliability for this group of teachers, with reliability estimates greater than 0.90. 
As expected, due to the fewer number of items, subscale reliability estimates were generally less than 
those for the composite scales; subscale reliability estimates ranged from a low of 0.76 for Assessment 
and Monitoring to a high of 0.91 for Communication of Mission. Nonetheless, all subscales evidenced 
either a standard error of measurement of less than 4% or a reliability estimate in excess of 0.75. Thus, for 
research purposes with this group of respondents, each subscale appears to provide reasonably stable 
information about the construct measured. 

To verify that items were eliciting responses consistent with the scales’ constructs for this group of 
respondents, item level statistics reflecting the correlation of item response to subscale and scale totals 
were examined. Any item evidencing a correlation less than 0.20 was flagged for additional review (see 
items marked in bold in Table A-3 in the Appendix). These are potentially aberrant items that can reduce 
both the reliability and validity of the scales. As shown in Table 1, one Teacher Survey item (in the 
Redesign Organization subscale associated with Leadership) was flagged for review; a decision about 
continued use will be made in view of content appropriateness. 
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Table 1. Reliability Characteristics of Survey for Teachers (Wave I) 



Scale 


No. of 
Items 


No. of 

Review Items 


Reliability* 

(Coefficient 

Alpha) 


s.e.m.* (%) 


School Environment 


30 


0 


0.91 


5.18 (1.9) 


Parent Involvement 


7 


0 


0.82 


2.27 (3.6) 


Academic Press 


8 


0 


0.84 


2.28 (3.2) 


Safe and Orderly Climate 


7 


0 


0.84 


2.28 (3.6) 


Assessment and Monitoring 


6 


0 


0.76 


2.66 (3.7) 












Leadership 


24 


1 


0.95 


3.88 (1.8) 


Communication of Mission 


7 


0 


0.91 


1.73 (2.7) 


Manage Instruction 


8 


0 


0.89 


1.90 (3.0) 


Redesign Organization 


10 


1 


0.86 


2.67 (3.0 












Professional Community 


24 


0 


0.91 


4.03 (1.9) 


Collaboration 


8 


0 


0.82 


2.17(3.0) 


Professional Development 


8 


0 


0.90 


1.91 (2.6) 


Support for Teacher Influence 


8 


0 


0.88 


2.10(2.9) 












Instruction 


26 


0 


0.91 


4.42 (1.19) 


Individualization 


9 


0 


0.81 


2.64 (3.3) 


Structure 


8 


0 


0.84 


2.15(3.0) 


Opportunity to Learn 


9 


0 


0.90 


2.14(2.6) 



*Five response possibilities for n=449 Wave I teacher respondents on item’s nine-category z-score continuum 



Principal survey reliability and validity. Information about the reliability of the principal survey results 
for the 19 principal respondents in Wave I is provided in Table 2. Because reliability focuses on random 
variability in the data and there were only 19 principal respondents, these reliability values may 
underestimate those that will result with use of the survey across the entire sample. Likewise, the number 
of potentially aberrant items is likely an overestimate. Even so, all scales and subscales evidenced 
estimates for the standard error of measurement of less than 5%. 

Item level statistics arc provided in Table A-4 in the Appendix. Any item evidencing a correlation less 
than 0.20 was flagged for additional review. As shown in Table 2 below, 15 such items were flagged and 
are under review; a decision about continued use will be made in view of content appropriateness. For 
example, in Table A-4 in the Appendix, under School Environment, Item 1-3 with a subscale correlation 
of .04 has been flagged and is marked in bold. This item, included in the Parent involvement set of items, 
is about parental support (“My school views strong parental support as an important condition for school 
effectiveness”)- In contrast, items in the same set with high subscale correlation coefficients (i.e., item 1-4 
with a correlation coefficient of .58 and Item 1-5 with an item correlation coefficient of .69) are about 
parent participation (i.e., “My school has a specific parent involvement initiative that encourages parents 
to participate in decisions about school practices” and “Parents frequently visit my office with their 
questions and concerns”). Content comparisons such as these, between items with high and low subscale 
correlation coefficients, are being used to clarify subcomponent substance and make decisions about 
continued use of flagged items. 
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Table 2. Reliability Characteristics of Survey for Principals (Wave I) 



Scale 


No. of 
Items 


No. of 

Review Items 


Reliability* 

(Coefficient 

Alpha) 


s.e.m.* (%) 


School Environment 


30 


11 


0.76 


5.04 (1.9) 


Parent involvement 


8 


4 


0.56 


2.88 (4.0) 


Academic Press 


8 


2 


0.51 


2.45 (3.4) 


Safe and orderly climate 


7 


1 


0.68 


2.19(3.5) 


Assessment and Monitoring 


7 


2 


0.72 


1.99 (3.2) 












Leadership 


25 


4 


0.89 


4.64 (2.1) 


Communication of Mission 


7 


1 


0.83 


2.09 (3.3) 


Manage instruction 


8 


0 


0.80 


2.29 (3.2) 


Redesign organization 


10 


3 


0.64 


3.30 (3.7) 



*Five response possibilities for n=19 Wave I principal respondents on item’s nine-category z-score continuum 



Scale coherence. To examine subscale coherence, items in each subscale that correlated more highly with 
the overall set of items than with the designated subscale were flagged for further review. The patterns of 
correlations between subscales and overall scales were examined in order to identify any subscales that 
appeared to function in an unexpected manner for Wave I respondents. A general expectation was that 
subscales should be more highly correlated to the overall scale to which it is assigned than to the other 
overall scales in the survey. This review is preliminary to a more formal confirmatory factor analysis that 
will be completed when Wave II data become available. Tables 3 and 4 summarize the patterns of 
relationships of the subscales with the overall scales and the percentage of shared variance for Wave I 
teachers and principals, respectively. As shown in Table 3, the patterns of relationships between scales 
and subscales in the teacher survey are generally consistent with that expected for coherent and unique 
scales. Specifically, the correlations of overall scales to the respective subscales were strong, ranging 
from approximately 0.47 to 0.93, and subscales exhibited noticeably lower correlations with other overall 
scales within the survey. 



Table 3. Relationships between Teacher Scales and Subscales (Wave I) 







Correlations 






Percent Variance Shared 




Environ 


Communit 


Lead 


Instruc 


Environ 


Communit 


Lead 


Instruc 


Environ 


1.00 


0.67 


0.78 


0.52 


Environ 


100.0 


44.7 


61.3 


26.6 


Communit 


0.67 


1.00 


0.77 


0.53 


Communit 


44.7 


100.0 


59.7 


28.0 


Lead 


0.78 


0.77 


1.00 


0.47 


Lead 


61.3 


59.7 


100.0 


22.2 


Instruc 


0.52 


0.53 


0.47 


1.00 


Instruc 


26.6 


28.0 


22.2 


100.0 


Parlnvlv 


mm 


0.55 


0.61 


0.36 


Parlnvlv 


60.2 


29.9 


37.6 


13.1 


AcadPres 


M 


0.54 


0.67 


0.38 


AcadPres 


70.6 


29.4 


45.4 


14.3 


OrdClim 


0.78 


0.50 


0.63 


0.35 


OrdClim 


61.5 


24.7 


39.8 


12.2 


Assmt&Mo 


0.75 


0.52 


0.54 


0.54 


Assmt&Mo 


55.5 


27.1 


29.6 


28.9 


Collab 


SEE 


0.78 


0.49 


0.43 


Collab 


19.6 


60.2 


23.9 


18.5 


ProfDev 




0.83 


0.55 


0.49 


ProfDev 


21.7 


68.4 


30.1 


24.1 


SupptTln 


HE 


0.80 


0.80 


0.35 


SupptTln 


47.6 


64.3 


64.8 


12.5 
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Table 3. Relationships between Teacher Scales and Subscales (Wave I) con’td 







Correlations 






Percent Variance Shared 




Environ 


Communit 


Lead 


Instruc 


Environ 


Communit 


Lead 


Instruc 


CommMiss 


mm 




0.90 


0.42 


CommMiss 


60.0 


47.0 


80.8 


17.3 


Manlnstr 


HI 


0.74 


0.92 


0.42 


Manlnstr 


46.8 


54.2 


84.0 


17.7 


Redesign 


0.70 


.070 


0.93 


0.45 


Redesign 


48.9 


49.5 


86.6 


20.5 


Individz 


mm 


0.43 


0.41 


0.77 


Individz 


22.3 


18.6 


17.0 


58.5 


Structur 


HH 


0.49 


0.45 


0.86 


Structur 


23.6 


24.2 


20.3 


74.5 


OpptLnr 


HI 


0.37 


0.29 


0.80 


OpptLnr 


9.7 


13.8 


8.7 


64.0 



The relationships shown in Table 4 are consistent with generally coherent and unique scales for both 
Environment and Leadership within the Wave I data for principals. Correlations of overall scales to the 
respective subscales ranged from approximately 0.50 to 0.90, and notably lower correlations of subscales 
were associated with the other overall scale. The one exception is Assessment and Monitoring within the 
Environment scale, which evidenced a higher correlation with the Leadership scale for the 19 principals 
responding in Wave I. This unintended relationship will be reexamined for stability with data that become 
available during Wave II. 



Table 4. Relationships between Principal Scales and Subscales (Wave I) 





Correlations 




Percent Variance Shared 


Environ 


Lead 


Environ 


Lead 


Environ 


1.00 


0.71 


Environ 


100.0 


50.6 


Lead 


0.71 


1.00 


Lead 


50.6 


100.0 


Parlnvlv 


0.56 


0.21 


Parlnvlv 


30.9 


4.2 


AcadPres 


0.87 


0.78 


AcadPres 


75.4 


30.1 


OrdClim 


0.65 


0.22 


OrdClim 


42.5 


4.7 


Assmt&Mo 


0.61 


0.76 


Assmt&Mo 


37.8 


58.2 


CommMiss 


0.68 


0.87 


CommMiss 


46.1 


76.5 


Manlnstr 


0.70 


0.91 


Manlnstr 


48.3 


83.0 


Redesign 


0.49 


0.83 


Redesign 


24.4 


69.0 



CURRENT STATUS AND NEXT STEPS 

To summarize, a model was developed to conceptualize how high-needs schools are organized for 
academic success based on a review and integration of four research literatures in 2003. The model and a 
research project to examine the model were proposed (Apthorp, 2003). External reviewers of the proposal 
affirmed the soundness of the design and suggested some revisions in particular model subcomponents 
and sampling procedures. These suggestions were incorporated in model and instrument development and 
sampling procedures conducted in 2004. 

In 2004, teacher and principal surveys were developed and pools of HP and LP high-needs elementary 
schools were identified in eleven states. Initially, study participants were recruited in five of the states; a 
second recruitment effort was later conducted in the remaining six states. This resulted in two waves of 
data collection: Wave I from March 1 to June 30, 2004 and Wave II from August 23 to October 31, 2004. 
Survey respondents in both waves answered questions about activities and practices that occurred in their 
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schools during the 2003-2004 academic year. Survey data from respondents in Wave II not present in the 
school during the 2003-2004 year were excluded from all analyses. 

Also in 2004, using the Wave I survey data, the research team examined measurement viability of the 
survey scales and subscales for representing model components and subcomponents. Wave I 
measurement viability analyses suggest that the teacher survey is viable for subsequent model analyses. 
Analyses are planned for examination of its continued viability once the Wave II survey data become 
available. Flagged items in both the teacher and principal Surveys are being reviewed for content 
appropriateness before they are included in the final scales. Also, once additional data are available from 
Wave II, the functional characteristics of the principal survey can be considered with greater assurance. 

Analyses also are planned to examine whether or not it is appropriate to combine Wave I and II data. The 
demographic comparability of the HP and LP groups of schools also will be examined. Differences 
between the HP and LP groups will be flagged and adjustments in subsequent analyses will be made to 
account for effects associated with these differences. Sample size sufficiency will also be examined. 

Subsequent data analyses are planned for examining the adequacy of the model. The model posited is 
complex in terms of the number of variables being measured and the different sources from which data 
are collected (i.e., principals and teachers). As planned, these data will be analyzed using multiple group, 
multilevel modeling. Using MPlus, this type of analysis accounts for the nested structure of the data while 
acknowledging relationships among variables (Muthen & Muthen, 2003). First, using the combined data 
from the HP and LP schools and Mplus structural equation modeling (SEM), evidence will be generated 
to answer the question, to what degree and in what manner are the model components related? Second, a 
chi-square test of model fit will be used to determine if the model is appropriate for both HP and LP 
schools by running Mplus SEM with the HP/LP identifier. Estimates/s. e. (weights of connections) will be 
examined to learn how the HP and LP models differ. The third stage of the model analysis involves 
unpacking the model. By using Mplus SEM and removing the fixed operational subcomponents of each 
component, it will be determined if additional variance can be explained. Finally, we will examine 
whether the unpacked model is different for HP and LP schools and examine estimates/s. e. (weights of 
connections) to learn how the unpacked HP and LP models differ. 

While the posited model of how high needs schools are organized for success is research-based, it is not 
yet evidence-based. Results of the model analyses will confirm or disconfirm the adequacy of the model 
components, subcomponents, and relationships. The research team and others at McREL will use the 
results to develop or amend tools and guidance for educators when deciding which school practices and 
activities to pursue for greatest impact on student achievement. 
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APPENDIX 



Table A-l 

Table A-2 
Table A- 3 
Table A-4 



Correlations between Math and Reading Assessments Used 
to Identify High-Performing and Low-Performing, High-Needs Schools 

Number of Schools Identified for Each Sample Pool 

Item Performance Statistics for Teacher Survey (Wave 1, n = 449) 

Item Performance Statistics for Principal Survey (Wave 1, n = 19) 



A-l 




Table A-l. Correlations between Math and Reading Assessments Used 



to Identify High-Performing and Low-Performing, High-Needs Schools 



State 


Correlations* 


Between Reading and 
Math ’00 (Zscore) 


Between Reading and 
Math ’01 (Zscore) 


Between Reading and 
Math ’02 (Zscore) 


Colorado 


.848 


.866 


.851 


Georgia 




.906 


.920 


Kansas 


.728 


.784 


.788 


Michigan 


.851 


.875 


.823 


Missouri 


.547 


.561 


.684 


New Jersey 


.922 


.908 


.921 


Ohio 


.917 


.930 


.915 


Oregon 


.850 


.808 


.890 


Texas 


.889 


.887 


.899 


Virginia 


.855 


.896 


.856 



* All correlations significant at the .01 level. 



A-2 






Table A-2. Number ol 


f Schools Identified for I 


Each Sample Pool 


Time of 




Number of Schools in Each Sample Pool 


Data Collection 


State 


HP 


LP 


Wave I 


Ohio 


68 


79 


(Mar 1 - Jun 30, 
2004) 


Oregon 


32 


34 


Texas 


291 


245 


Michigan 


79 


73 




Minnesota 


17 


17 


Subtotal 


487 


448 


Wave II 


Colorado 


25 


27 


(Aug 23 - Oct 31, 
2004) 


Kansas 


23 


22 


Georgia 


71 


84 


Virginia 


23 


51 




Missouri 


54 


51 




New Jersey 


56 


55 




Subtotal 


252 


290 




Total 


739 


738 



A-3 





Table A-3. Item Performance Statistics for Teacher Survey (Wave 1, n=449) 



School Environment 



Res = 


1 


2 


3 


4 


5 


6 


7 


8 


9 


other 


pol. 


mean 


s.d. 


subscale 

cor. 


Item 1-1 






18% 


13% 


41% 


28% 










+ 


4.78 


1.05 


0.53 


Item 1-2 


i% 


3% 


9% 




49% 


38% 


0% 








+ 


5.11 


1.04 


0.51 


Item 1-3 


0% 


4% 


7% 


26% 




62% 


0% 








+ 


5.08 


1.28 


0.47 


Item 1-4 


i% 


3% 


6% 




40% 


50% 


0% 








+ 


5.25 


1.05 


0.50 


Item 1-5 


i% 


4% 




23% 




71% 


1% 








+ 


5.35 


1.15 


0.53 


Item 1-6 




6% 


6% 




31% 


57% 


0% 








+ 


5.26 


1.16 


0.44 


Item 1-7 




2% 


11% 


19% 


43% 


25% 


1% 








+ 


4.81 


1.01 


0.49 


Item 1-8 


2% 


1% 




22% 


74% 




1% 








+ 


4.69 


0.74 


0.47 


Item 1-9 


0% 


3% 




23% 




73% 








i% 


+ 


5.39 


1.07 


0.56 


Item 1-9 




3% 


3% 


27% 




66% 


0% 






i% 


+ 


5.26 


1.12 


0.53 


Item 1-11 


1% 




7% 


5% 


47% 


41% 


0% 








+ 


5.18 


0.92 


0.51 


Item 1-12 




2% 


4% 


25% 




67% 


0% 






i% 


+ 


5.30 


1.08 


0.53 


Item 1-13 


2% 


4% 




18% 


76% 




0% 








+ 


4.65 


0.82 


0.57 


Item 1-14 


0% 


4% 


5% 


25% 




65% 


0% 








+ 


5.19 


1.21 


0.57 


Item 1-15 


2% 




6% 


23 % 




68% 


1% 






0% 


+ 


5.27 


1.20 


0.58 


Item 1-16 


2% 


4% 


5% 


30% 




59% 


1% 








+ 


5.04 


1.30 


0.51 


Item 1-17 


1% 


6% 


4% 




31% 


58% 


0% 








+ 


5.28 


1.18 


0.58 


Item 1-18 




4% 


10% 


7% 


43% 


35% 


1% 








+ 


4.97 


1.09 


0.55 


Item 1-19 


2% 




6% 


29% 




63% 


0% 








+ 


5.16 


1.19 


0.54 


Item 1-20 




2% 


5% 


23% 




68% 


1% 






i% 


+ 


5.29 


1.12 


0.57 


Item 1-21 


3% 








96% 








1% 




+ 


4.90 


0.82 


0.23 


Item 1-22 


1% 


3% 


4% 


18% 




73% 


0% 








+ 


5.32 


1.21 


0.56 


Item 1-23 




4% 


11% 




84% 




1% 






0% 


+ 


4.68 


0.84 


0.27 


Item 1-24 




1% 


6% 


28% 




64% 


1% 






1% 


+ 


5.23 


1.09 


0.35 


Item 1-25 


1% 


4% 




27% 




68% 


0% 








+ 


5.25 


1.20 


0.32 


Item 1-26 


2% 




6% 


19% 




71% 


1% 








+ 


5.31 


1.22 


0.25 


Item 1-27 




2% 


7% 


9% 


40% 


43% 










+ 


5.14 


0.96 


0.52 


Item 1-28 


2% 




6% 


26% 




66% 


1% 






0% 


+ 


5.25 


1.17 


0.41 


Item 1-29 




2% 


6% 


29% 




62% 


0% 






0% 


+ 


5.15 


1.16 


0.52 


Item 1-30 


2% 




3% 


24% 




70% 


1% 






0% 


+ 


5.32 


1.17 


0.58 



Professional Community 



Res = 


1 


2 


3 


4 


5 


6 


7 


8 


9 


other 


pol. 


mean 


s.d. 


subscale 

cor. 


Item 2-1 




2% 


6% 


23% 


32% 


28% 


0% 






8% 


+ 


4.87 


0.98 


0.52 


Item 2-2 




2% 


6% 


18% 


31% 


33% 


0% 






10% 


+ 


4.98 


0.97 


0.54 


Item 2-3 




1% 


10% 


18% 


28% 


37% 


0% 






6% 


+ 


4.96 


1.03 


0.50 


Item 2-4 






12% 


14% 


17% 


16% 


10% 






30% 


+ 


4.98 


1.09 


0.47 


Item 2-5 






4% 


34% 


27% 


25% 


0% 






9% 


+ 


4.84 


0.86 


0.53 


Item 2-6 






11% 


6% 


15% 


34% 


0% 






34% 


+ 


5.07 


0.92 


0.35 


Item 2-7 


0% 


3% 




12% 


35% 


46% 


0% 






2% 


+ 


5.23 


0.94 


0.46 


Item 2-8 






10% 


8% 


14% 


38% 


1% 






29% 


+ 


5.12 


0.94 


0.36 


Item 3-1 




2% 


4% 


21% 


32% 


41% 










+ 


5.04 


0.99 


0.59 


Item 3-2 




4% 


5% 


28% 


32% 


31% 


1% 








+ 


4.84 


1.06 


0.60 


Item 3-3 






6% 


36% 


29% 


29% 


0% 








+ 


4.82 


0.93 


0.60 


Item 3-4 






6% 


18% 


30% 


44% 


1% 








+ 


5.16 


0.95 


0.60 


Item 3-5 






8% 


12% 


53% 


27% 


0% 








+ 


5.00 


0.85 


0.64 


Item 3-6 




3% 


5% 


25% 


35% 


31% 


1% 








+ 


4.89 


1.03 


0.61 


Item 3-7 






12% 


15% 


26% 


47% 


0% 








+ 


5.09 


1.05 


0.54 


Item 3-8 






9% 


11% 


49% 


29% 


1% 








+ 


5.02 


0.91 


0.50 


Item 4-1 






18% 


5% 


38% 


39% 


0% 








+ 


4.98 


1.08 


0.48 


Item 4-2 




2% 


9% 


10% 


37% 


41% 


0% 








+ 


5.05 


1.05 


0.55 


Item 4-3 






9% 


16% 


45% 


30% 


0% 








+ 


4.97 


0.92 


0.49 


Item 4-4 




4% 


10% 


12% 


37% 


37% 










+ 


4.93 


1.11 


0.57 


Item 4-5 






5% 


28% 


37% 


29% 


0% 








+ 


4.91 


0.89 


0.64 


Item 4-6 




i% 


6% 


11% 


40% 


41% 


1% 








+ 


5.18 


0.92 


0.55 


Item 4-7 


0% 


2% 


7% 


31% 




58% 


1% 








+ 


5.06 


1.23 


0.52 


Item 4-8 




1% 


6% 


8% 


38% 


46% 


1% 








+ 


5.23 


0.93 


0.53 



1 



Leadership 



Res = 


1 


2 


3 


4 


5 


6 


7 


8 


9 


other 


pol. 


mean 


s.d. 


subscale 

cor. 


Item 7-1 




2% 


8% 


10% 


47% 


33% 










+ 


4.99 


0.98 


0.61 


Item 7-2 




1% 


7% 


6% 


37% 


48% 










+ 


5.23 


0.95 


0.77 


Item 7-3 


0% 


4% 


9% 




37% 


50% 


0% 








+ 


5.20 


1.11 


0.69 


Item 7-4 


0% 


6% 




8% 


30% 


54% 


i% 








+ 


5.28 


1.09 


0.65 


Item 7-5 




1% 


7% 


15% 


37% 


40% 


i% 








+ 


5.08 


0.98 


0.73 


Item 7-6 


i% 


6% 




8% 


32% 


53% 


0% 








+ 


5.24 


1.11 


0.71 


Item 7-7 




1% 


6% 


11% 


36% 


46% 


1% 








+ 


5.22 


0.92 


0.73 


Item 7-8 




1% 


6% 


17% 


31% 


45% 










+ 


5.12 


0.97 


0.50 


Item 7-9 




3% 


6% 


12% 


36% 


41% 


1% 








+ 


5.09 


1.06 


0.74 


Item 7-9 




2% 


12% 


9% 


40% 


37% 


0% 








+ 


4.99 


1.06 


0.60 


Item 7-11 






16% 


15% 


34% 


35% 


0% 








+ 


4.89 


1.06 


0.69 


Item 7-12 




3% 


9% 


13% 


37% 


37% 


0% 








+ 


4.97 


1.08 


0.78 


Item 7-13 




3% 


10% 


13% 


36% 


37% 


0% 








+ 


4.96 


1.09 


0.73 


Item 7-14 




2% 


8% 


15% 


36% 


38% 


1% 








+ 


5.01 


1.05 


0.70 


Item 7-15 




3% 


6% 


17% 


38% 


35% 


1% 








+ 


4.98 


1.04 


0.59 


Item 7-16 






15% 


16% 


33% 


36% 


0% 








+ 


4.91 


1.06 


0.04 


Item 7-17 


0% 


2% 


8% 


37% 




52% 


1% 








+ 


4.93 


1.22 


0.60 


Item 7-18 


2% 


3% 


9% 




30% 


57% 


0% 








+ 


5.25 


1.17 


0.65 


Item 7-19 




1% 


8% 


37% 




54% 


0% 






0% 


+ 


5.01 


1.14 


0.68 


Item 7-20 




1% 


8% 


17% 


42% 


31% 


0% 








+ 


4.95 


0.96 


0.71 


Item 7-21 




3% 


8% 


22% 


33% 


34% 


1% 








+ 


4.90 


1.07 


0.63 


Item 7-22 


1% 


2% 


12% 




40% 


44% 


1% 








+ 


5.11 


1.12 


0.64 


Item 7-23 




1% 


7% 


15% 


35% 


41% 


0% 








+ 


5.08 


0.99 


0.76 


Item 7-24 




2% 


6% 


14% 


33% 


45% 


1% 








+ 


5.17 


1.00 


0.57 


1 nstruction 




Res = 


1 


2 


3 


4 


5 


6 


7 


8 


9 


other 


pol. 


mean 


s.d. 


subscale 

cor. 


Item 8-1 


1% 


i% 




31% 




65% 


0% 








+ 


5.26 


1.12 


0.36 


Item 8-2 




2% 




23% 




75% 


1% 






0% 


+ 


5.49 


0.95 


0.42 


Item 8-3 


2% 




3% 


33% 




61% 


1% 








+ 


5.16 


1.18 


0.49 


Item 8-4 


1% 


3% 


6% 


28% 




61% 


0% 








+ 


5.07 


1.26 


0.45 


Item 8-5 


2% 




18% 




80% 






0% 






+ 


4.60 


0.91 


0.43 


Item 8-6 


2% 




4% 


28% 




66% 


0% 






0% 


+ 


5.24 


1.16 


0.39 


Item 8-7 




1% 


21% 




76% 




2% 






0% 


+ 


4.57 


0.91 


0.40 


Item 8-8 




2% 


5% 


15% 


40% 


37% 


0% 








+ 


5.06 


0.97 


0.46 


Item 8-9 




4% 


8% 


15% 


35% 


37% 


1% 








+ 


4.97 


1.11 


0.43 


Item 9-1 




0% 


6% 


21% 


37% 


35% 


1% 








+ 


5.01 


0.94 


0.41 


Item 9-2 


0% 


1% 




19% 


43% 


35% 


1% 








+ 


5.13 


0.86 


0.60 


Item 9-3 




1% 


7% 


37% 




54% 


1% 






0% 


+ 


5.02 


1.15 


0.50 


Item 9-4 


0% 


1% 


17% 




43% 


37% 


1% 








+ 


5.01 


1.11 


0.59 


Item 9-5 




1% 


4% 


24% 


44% 


26% 


1% 








+ 


4.91 


0.88 


0.56 


Item 9-6 




1% 


5% 


23% 


42% 


28% 


1% 








+ 


4.92 


0.92 


0.65 


Item 9-7 




2% 


6% 


26% 


41% 


25% 


1% 








+ 


4.84 


0.96 


0.57 


Item 9-8 


0% 


3% 




15% 


45% 


35% 


2% 








+ 


5.14 


0.95 


0.51 


Item 10-1 






8% 


24% 


37% 


22% 


8% 


i% 






+ 


5.00 


1.08 


0.53 


Item 10-2 






5% 


39% 


38% 


17% 


2% 








+ 


4.72 


0.85 


0.60 


Item 10-3 






12% 


16% 


31% 


28% 


12% 








+ 


5.12 


1.19 


0.58 


Item 10-4 






10% 


12% 


31% 


47% 


i% 








+ 


5.17 


0.99 


0.54 


Item 10-5 






13% 


22% 


31% 


26% 


7% 


i% 






+ 


4.95 


1.16 


0.61 


Item 10-6 






11% 


15% 


26% 


47% 


0% 








+ 


5.10 


1.04 


0.52 


Item 10-7 






4% 


36% 


35% 


24% 


1% 








+ 


4.81 


0.88 


0.57 


Item 10-8 






14% 


26% 


36% 


23% 


1% 








+ 


4.73 


1.01 


0.56 


Item 10-9 






7% 


11% 


57% 


22% 


2% 








+ 


5.01 


0.84 


0.49 
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Table A-4. Item Performance Statistics for Principal Survey (Wave 1, n=19) 



School Environment 



Res = 


1 


2 


3 


4 


5 


6 


7 


8 


9 


other 


pol. 


mean 


s.d. 


subscale 

cor. 


Item 1-1 






16% 


11% 


26% 


47% 










+ 


5.05 


1.10 


0.58 


Item 1-2 




5% 




42% 




53% 










+ 


4.95 


1.19 


0.43 


Item 1-3 








16% 


79% 




5% 








+ 


4.95 


0.60 


0.04 


Item 1-4 




5% 




32% 




63% 










+ 


5.16 


1.18 


0.14 


Item 1-5 






11% 


42% 




47% 










+ 


4.84 


1.14 


0.69 


Item 1-6 






21% 






79% 










+ 


5.37 


1.22 


- 0.12 


Item 1-7 






21% 






79% 










+ 


5.37 


1.22 


0.12 


Item 1-8 




5% 




11% 


42% 


42% 










+ 


5.16 


0.99 


0.43 


Item 1-9 




11% 






89% 












+ 


4.68 


0.92 


- 0.27 


Item 1-10 




11% 






89% 












+ 


4.68 


0.92 


0.29 


Item 1-11 






16% 




84% 












+ 


4.68 


0.73 


0.59 


Item 1-12 








32% 




68% 










+ 


5.37 


0.93 


0.32 


Item 1-13 


5% 








95% 












+ 


4.79 


0.89 


0.00 


Item 1-14 






21% 






79% 










+ 


5.37 


1.22 


0.46 


Item 1-15 






16% 




84% 












+ 


4.68 


0.73 


0.29 


Item 1-16 




11% 




11% 


79% 












+ 


4.58 


0.94 


0.39 


Item 1-17 








32% 




68% 










+ 


5.37 


0.93 


0.32 


Item 1-18 








16% 




74% 


11% 








+ 


5.79 


0.83 


0.33 


Item 1-19 






5% 


37% 




53% 


5% 








+ 


5.16 


1.14 


0.42 


Item 1-20 






26% 






74% 










+ 


5.21 


1.32 


0.64 


Item 1-21 








32% 




68% 










+ 


5.37 


0.93 


0.32 


Item 1-22 










100% 












+ 


5.00 


0.00 


0.00 


Item 1-23 


5% 








95% 












+ 


4.79 


0.89 


0.57 


Item 1-24 




5% 




32% 




63% 










+ 


5.16 


1.18 


0.64 


Item 1-25 










100% 












+ 


5.00 


0.00 


0.00 


Item 1-26 






16% 




84% 












+ 


4.68 


0.73 


0.54 


Item 1-27 


5% 








95% 












+ 


4.79 


0.89 


0.63 


Item 1-28 






5% 




89% 




5% 








+ 


5.00 


0.65 


- 0.04 


Item 1-29 




11% 






32% 


58% 










+ 


5.26 


1.21 


0.72 


Item 1-30 






11% 




42% 


47% 










+ 


5.26 


0.91 


0.39 



Leadership 



Res = 


1 


2 


3 


4 


5 


6 


7 


8 


9 


other 


pol. 


mean 


s.d. 


subscale 

cor. 


Item 2-1 




5% 




32% 




63% 










+ 


5.16 


1.18 


0.82 


Item 2-2 




5% 




26% 




68% 










+ 


5.26 


1.16 


0.49 


Item 2-3 






16% 




84% 












+ 


4.68 


0.73 


0.14 


Item 2-4 






11% 


21% 




68% 










+ 


5.26 


1.12 


0.85 


Item 2-5 






11% 


37% 




53% 










+ 


4.95 


1.15 


0.60 


Item 2-6 






16% 




84% 












+ 


4.68 


0.73 


0.58 


Item 2-7 




5% 


5% 




37% 


53% 










+ 


5.26 


1.07 


0.60 


Item 2-8 






5% 


16% 


32% 


47% 










+ 


5.21 


0.89 


0.62 


Item 2-9 








47% 




53% 










+ 


5.05 


1.00 


0.37 


Item 2-10 








32% 




68% 










+ 


5.37 


0.93 


0.62 


Item 2-11 








32% 




68% 










+ 


5.37 


0.93 


0.33 


Item 2-12 




5% 




16% 


47% 


32% 










+ 


5.00 


0.97 


0.46 


Item 2-13 




11% 






32% 


58% 










+ 


5.26 


1.21 


0.54 


Item 2-14 




5% 




37% 




58% 










+ 


5.05 


1.19 


0.75 


Item 2-15 






16% 




84% 












+ 


4.68 


0.73 


0.45 


Item 2-16 






26% 






74% 










+ 


5.21 


1.32 


0.17 


Item 2-17 




5% 


5% 




21% 


63% 


5% 








+ 


5.47 


1.14 


- 0.06 


Item 2-18 






21% 






79% 










+ 


5.37 


1.22 


0.31 


Item 2-19 


5% 






16% 


79% 












+ 


4.63 


0.93 


0.55 


Item 2-20 






16% 




84% 












+ 


4.68 


0.73 


0.46 


Item 2-21 




5% 




37% 




58% 










+ 


5.05 


1.19 


0.38 


Item 2-22 






21% 






79% 










+ 


5.37 


1.22 


- 0.19 


Item 2-23 






26% 






74% 










+ 


5.21 


1.32 


0.57 


Item 2-24 




5% 




21% 




74% 










+ 


5.37 


1.13 


0.51 


Item 2-25 




5% 






37% 


53% 


5% 








+ 


5.47 


0.99 


0.72 
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